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CHAPTER 2 



ASSESSMENT OF PRACTICAL SKILLS 
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2.1 Aim 

The aim of the work presented in this section of the thesis is to study 

the reliability and validity of assessment. 

Assessment of a manipulative skill of a pupil carried out by a number of 
judges using a check-list is ideally expected to be the same. However, 
one may ask, in reality, if that is so! The nature of 
agreement/ disagreement among a number of judges is worth 
considering in order to answer such questions and to understand the 
reliability of assessment. 

During the assessment of a set of pupils by a teacher, as is usually the 
case in real life, more than one skill may have to be assessed: One may 
ask if it would be less reliable when a judge concentrates on assessing 
more than one skill, instead of assessing only one skill. Here attempts 
have been made to find answers to such a question. 

During a practical session, pupils carry out manipulative skills in order 
to collect data/record observations/ deduce inferences which are 
submitted as written reports. As the task of assessing the skills by on- 
the-spot observation during practical sessions (i.e. Method -T) is time- 
consuming and thus difficult for a teacher, compared with the 
assessment carried out by checking the written report (i.e. Method -P) 
which can be done at home, it may be worthwhile to see if there is a 
significant correlation between the standard of achievement of a pupil 
assessed in both ways. So a comparison between these two methods of 
assessment is also undertaken. This will help us to find out if the result 
of assessment of a pupil by Method -P can be used as a measure of their 
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performance in executing the manipulative skills during a practical 

session. ~ , 

2.2 Development of the study materials - Preliminary runs 

i) Method 

Attempts were made to design an experiment which will allow us to (a) 
assess on the spot by observation of the performance of pupils in 
carrying out the skill of measuring a certain volume of liquid using a 
measuring cylinder, (b) assess by marking the final results of the whole 
experiment, and (c) study the reliability and validity of assessment of 
the skill. 

. Having given some thought, it was decided that the following 
experiment "reaction of Mg with diLlS"^;^. ...,/. 
experiment suitable for this purpose and would satisfy the scope and 

limitations described earlier. 

> 
■ Prior to the final run, a trial run was carried out by using about 25 pupils 
of an age range of 13-14 years from Norwich School, The Close, 
Norwich. The objective of this trial run was to find out the following: 

(a) If the work-sheet is self-explanatory and easy to follow by all 

the pupils. 

(b) Does the method work or is there 1 any need for further 

modification? 

(c) How many runs (number of experiments) can be carried out 

within a definite time limit (say one hour)? 

(d) Are the apparatus and chemicals adequate for the work? 

(e) How many pupils can be assessed by each judge for the 
measurement of volume and pouring out the liquid? 
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This trial run helped me to eliminate any deficiency in the proration of 

- - • r 

the instruction in the work-sheet and also the arrangement of apparatus 

and chemicals. 

(it) Discussion of the out-come of the trial runs 

The length of the Mg ribbon was a problem. As it was 5 cm in length, 
when it was dropped into the flask containing dil. HC1, occasionally it 
fell horizontally and sank into the acid. However most of the time it 
dropped vertically and a part of it remained above the acid. So it was 
decided to cut the Mg ribbon into 2 parts, each of 2.5 cm length and the 
pupils were asked to use 2 pieces of Mg ribbon for each experiment. In 
this way the ribbon dropped horizontally into the flask and sank into 
the acid each time. No other modification was necessary. 



2.3 Final run 
(t) Method 

The tasks for the pupils were to find out the time taken by 2 pieces (2.5 
cm each piece) of Mg ribbon to dissolve completely in a mixture of dil. 
HC1 and water (see work-sheet 2.1 in Appendix 1.1). 

Four sets of 4th form pupils (14-15 year old) from a local comprehensive 
school (Earlham School) took part in this experiment. About 20 pupils 
(boys and girls) were in each set and they were randomly allocated a 
work-station before the work-session. 

One large teaching laboratory in the School of Chemical Sciences at the 
University of East Anglia was used for this purpose. Each work-station 
was numbered so that each pupil, randomised at the door and provided 
with numbered labels, could come quickly to the matching numbered 
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desk. 



In order to facilitate the on-the-spot assessment by the judges who each 
carried a check-list and identified the pupils only by their numbers, the 
following arrangements were made. The pupils carried identifying 
number labels in both of their arms and the number plates at the work- 
stations were raised to a reasonable height so that the judges could see it 
clearly from a reasonable distance. 

Each session lasted for about an hour. In between two sessions, there 
was an interval of about 15-30 minutes during which the used glass- 
ware was removed and new chemical and ; apparatus were laid out. A 
list of chemicals and apparatus used in this experiment is given in 
Appendix 1.2. 

> 
Each pupil was asked to carry out 6 runs (experiments) using 6 acid- 
water mixtures of different volume ratios. Work-sheets and other 
required materials were supplied at the work-desk before the start of the 
experiment. 

Nine qualified science-teachers from local schools participated as judges. 
They were divided into 3 panels and were assigned for carrying out the 
on-the-spot assessment by observation of the performance of the skill in 
the following way: 



ft- 



M 





Panel 




A 


B 


C 


M + P 


M + P 


M + P 


M 


P 


M + P 


- M + P 


M 


P 


P 


M + P 


M 



Session_ 

(trial run) 
1 

2 

U and P stand for measuring volume by a cylinder and pouring the 
liquid out of the cylinder respectively. 

the experiment carried out by the first set of pupils was .used by-the 
judges as a trial run to gain a clear idea of the nature of their task in thrs 
experiment. This was helpful for their asessment work in the three 

subsequent sessions. > 

Before the start of the experiment, the judges were briefed about the 
nature of the task and clear instructions were given about the job in 
each session, as has been outlined in the table above. 

It was thought that, among all the skills involved in this experiment, 
*e two most important skills to affect the final results, i.e. the time for 
Mg to dissolve, are: (1) measuring the volume of acid/acid + water 
mixture, and (2) pouring out the liquid into the flask. Each judge 
assessed each pupil once for measurement of volume (M) or pouring 
out the liquid <P) or both (M) and (P), according to the work-scheme 
given earlier. The judges were instructed to give 1" mark for correctly 
executing the skill and "0" for failing to do so. 
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The judges- checked -the accuracy of the volume measurement by taking 
a close look at the measuring cylinder and seeing if the lower meniscus 
was within 

+ 0.5 cm of the volume to be measured. The accuracy of the pouring 

skill was checked by looking at each pupil's effort to drain the last drop 
of liquid out of the cylinder. 

In order to assess on the basis of the outcome of the extended task, the 
results of the experiment submitted by the pupils in the form of a 
written report (i.e. the time taken for Mg to dissolve reported by the 
pupil) were checked against the values determined by me. For this : 
purpose each of the 6 experiments [see the work-sheet 2.1 in Appendix 
1.1] was repeated 8 times by me. The volume of acid was measured 
accurately, the pouring out operation was carried with great care and 
finally the exact time taken by the Mg ribbon to dissolve completely was 
recorded accurately. The mean value of time for each experiment (with 
the standard deviation as the accepted error limit) was used as the . 
standard result to carry out this assessment. A table containing these 
results and s.d. values is given in the Appendix 1.3. Probable 
explanation for the variation of the s.d values with the variation of 
acid:water composition is also given there. 
(ii) Results and Discussion 

As the first assessment session during the experiment was used as a trial 
run and referred to as session -0 in this section, results of only the 2nd, 
3rd and 4th sessions of the experiment will be used in the calculation 
here and they will be referred to hereafter as Session -1, Session -2 and 
Session -3 respectively. 
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According to the marking scheme, each pupil was marked in the 
following way: a panel of 3 judges marked only the volume 
measurement skill, another panel of 3 judges marked only the pouring 
out skill and the 3rd panel of 3 judges marked both the skills. As each 
judge was instructed to give 1 mark for correct execution of a skill, each 
pupil could get a maximum of 6 marks for volume measurement, and 6 
marks for pouring out the liquid. 



The data in the judges' assessment sheets for all the 3 sessions (see 
Apjendix 1.4) has been analysed and the information available in these 
tables was used to study the reliability and validity of assessment. .. 



In order to look at the reliability of assessment, the following two 
aspects have been considered: 

(i) comparison has been made between the achievement of the pupils 
for "their volume measurement skill assessed by a panel (say "A") of 3 
judges for the volume measurement alone, and that assessed by another 
panel (say "C") of 3 judges for both the volume measurement and 
pouring out the liquid. Similar comparison can be made between the 
achievement of the pupils for their liquid pouring skill assessed by a 
panel (say "B") of 3 judges for the pouring out liquid alone, and that 
assessed by the panel "C" for both volume measurement and pouring 
out liquid mentioned earlier. 



r 
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(ii) The possible ways in which a panel of 3 judges can mark a pupil 
during assessing a skill are shown in the table below: 



Possibilities of marks 
for a pupil from 3 judges 



Overall achievement 



of the pupil 



■ 


1 


1 


1 


3 marks 


.. . . 


1 


1 





2 marks 




1 








1 mark 


■ 








. 


mark 



Agreement 
a.'Vn.o'nffi 

the judges 
C Vesi 1 > 

No:0) 
1 


1 .' " 



When a pupil gets 3 marks (1 + 1 + 1) or "0" mark (0 + + 0), the 3 
judges in a panel are in perfect agreement, otherwise the judges are not 
in perfect agreement. A situation of perfect agreement among the 
judges is represented by "1", while the lack of perfect agreement is 
indicated by "0". 



Tables 2.1, 2.2, 2.3 and 2.4 contain summaries of the analysis of the data 
related to the reliability of the assessment studies. Tables 2.1 and 2.2, 
derived from further analysis of data in Table 2.6, 2.7 and 2.8, show the 
mean values of achievement when all the pupils in a session have been 
assessed for one skill, e.g. volume measurement/pouring out liquid, by 
a panel of judges, compared with the situation when both the skills 
have been assessed by another panel of judges. 



Tables 2.3 and 2.4, derived from further analysis of data in Table 2.9,2.10 



it:-- -j. 

F ■ 

r 
i 
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2.11, show the mean values of total agreement among the judges in a 
panel when all the pupils in a session have been assessed for one skill, 
e.g. measuring volume/pouring out liquid, by a panel of judges, 
^compared with the situation when both the skills have been assessed 
by another panel of judges. 

It appears that the mean value of the achievement for any set (or, for all 
the 3 sets together) for volume measurement does not differ 
significantly whether a panel of judges assessed it alone or another 
panel of judges assessed both skills of measuring volume and pouring a 
liquid. 

A similar conclusion can be drawn for the mean value of the 
achievement for pouring out skill, whether a panel of judges assessed 
that skill alone or another panel of judges assessed it together with the 
volume measurement skill. ^ 

The mean value of performance for the pouring out skill is much 
higher than that for the volume measurement skill. The results suggest 
that the pupils in every set have been able to carry out this "pouring out 
the liquid" skill very effectively, almost perfectly. However poor 
performance (about 33% of the total marks; for volume measurement 
suggests that the pupils would require further lessons in this skill. The 
final results of the experiments, which have been used as the basis for 
assessment by Method -P, are directly influenced by the pupils' ability to 
carry out the volume measurement skill efficiently. 

With regard to the agreement/disagreement among the judges, it can be 
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said from the Tables 2.3 and 2.4 that the judges are in total agreement for 
about 50% of the sample during the assessment of the volume 
measurement skill, irrespective of whether it was' assessed alone or 
assessed in conjunction with the pouring out skill. The amount of total 
agreement among the judges for assessing pouring out skill was about 
100%. As with the volume measurement skill, the pouring out skill 
showed no significant difference in the amount of total agreement 
depending on whether it was assessed alone or assessed with the 
volume measurement skill. 

For further analysis of the data of total agreement among the judges, 
several 2X3 ANOVA based on randomised groups were carried out. 
They were as follows: ■-; ■■.,.: .!; ■ < ,■; 

(i) 2 ways of assessing volume X 3 panels of 

measurement skill (A) judges (B) 

(ii) 2 ways of assessing pouring X 3 panels of 

out skill (A) judges (B) 

(ii) 2 ways of assessing volume X 3 sessions 

measurement skill (A) (sets of pupils)(B) 

(iv) 2 ways of assessing pouring X 3 sessions 

out skill (A) (sets of pupils)(B) 

Information derived from the ANOVA is in Table 2.5. 

None of the Fa value is significant even at p <0.05 level. This suggests 

that whichever way a skill, i.e. measurement of volume or pouring out 
liquid, is assessed, the amount of total agreement among the judges is 
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unaffected. 

This finding lends support to the earlier conclusion regarding the 
reliability of assessment based on the fact that the number of total 
agreements does not differ significantly between the two ways of assessing 
volume measurement or pouring out skill. 

Furthermore, as none of the Fg or F^b is significant even at p<^0.05, 
assessment being carried out by three panels of judges and in three 
sessions also do not have any significant effect on the number of 
agreement among the judges and hence on the reliability of the 
assessment. . 

The achievement based on the assessment by Method -T of performing 
each of the practical skills for all the pupils in each of the 3 sets working in 
3 sessions has been presented in Tables 2.6, 2.7 and 2.8. Appendix 1.4 
contains the "raw-data", i.e. 3 panels (A, B, C) of judges' mark-sheets for all 
the 3 assessment sessions. The data in each session were analysed in the 
following way. Marks obtained by each pupil for his/her volume 
measurement/liquid pouring out skill were added togehter and presented 
as the achievement data based on the assessment Method -T in Tables 2.6, 
2.7, 2.8. As each pupil was assessed by 3 judges for volume measurement 
alone, 3 judges for pouring out alone and 3 judges for both skills, the 
maximum marks that a pupil can get for correct execution of each skill is 
6. In each table, there is also a column of data showing the achievement 
of these pupils based on the assessment by Method -P which involves the 
judgement of the accuracy of the final results of the experiment (i.e. the 
time taken by the Mg ribbon to dissolve completely). A maximum of 6 
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marks can be obtained by a pupil for the correct results of 6 runs in 
experiment 2.1. 

Using judges' mark-sheets (see Appendix 1.4 J/Pupils' achievement data 
in Tables 2.6, 2.7, and 2.8, situations of agreement/disagreement among 
the judges in assessing the pupils in all 3 sessions have been analysed and 
are listed in Tables 2.9, 2.10 and 2.11. 



The marks awarded by the judges (assessment by method-T) for volume 
measurement and pouring out a liquid was plotted against the marks, 
based on the assessment by Methid -P, for all the pupils in the 3 sessions in 
Figures 2.1 and 2.2 respectively. Such a comparison can be used as a 
measure of validity of assessment. The data plotted in each figure appears 
to be widely scattered and it is quite difficult to distinguish any clear 
relationship (trend) between data gathered from two methods of 
assessment. For this Pearson-r (i.e. Pearson r product, moment corllation 
coefficient) values were calculated and the results are shown in Table 2.12. 
The Rvalues appear to lie within ± 0.5 suggesting the lack of a very strong 
(near to perfect) correlation. However, it is worthwhile to look at the 
statistical significance of the existing correlation as shown by the r-value. 

i 
Here the experimental hypothesis is one tailed and a positive correlation 
is expected. It is thought that a pupil's achievement for 
performing volume measurement skill/pouring out skill assessed by 
Method -T will be associated with the achievement of the 
pupil assessed by Method -P (i.e. marking the written report on the 
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outcome of the extended task of the whole experiment). According to the 
"Null Hypothesis" the correlation "f" value obtained from calculation 
could be due to chance factor; So the calculated "¥*" value is compared 
with the critical value of 'V' a t the corresponding degree of freedom df = 
58 (df = N-2, where N = no. of subjects in the experiment). 

One of the "ff' values calculated for the volume measurement skill ^ = 
0.306 (for the 6th run) appears to be higher than the critical value at p = 
0.025 but lower than the critical value at p = 0.01. This suggests that the 
calculated "<p' value is significant at p^0.025.1evel, i.e. % probability of 
correlation by chance factor is equal to or less than 2.5% and thus the 
"Null Hypothesis" is rejected and experimental hypothesis is accepted at p 
/0.025 level. However, the rest of the lh f" values calculated for volume 
measurement skill and all the jr f" values for pouring out skill seem to be 
lower than the lowest possible critical value of f (= 0.2306) for df = 58 
which is at p = 0.05. This suggests that the % probability of the correlation 
occurring by chance factor is higher than 5% and thus the "p" values are 
not significant statistically. 

A significant correlation, even though at p^0.025 level, shown (by the 
result of one of the runs of volume measurement experiment) between 
the achievement data of the pupils when assessed by method -T and that 
when assessed by method -P, suggests that the accuracy of the volume 
measurements do indeed have a strong influence on the final result, i.e. 
time needed for Mg ribbon to dissolve completely and assessment of the 
final result submitted as a written report can be used as a substitute for the 
assessment of volume measurement skill on the spot. 
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Although the "pouring out liquid" skill is an important technique in the 
whole experiment and should have a strong influence on the final result, 
the lack of a strong correlation between the achievement data from 
method -T and that from method -P is likely to be due to the fact that 
performance of both the volume measurement skill and pouring out skill 
affect the accuracy of the final result and hence the pupils' achievement in 
assessment by method -p. From the discussion of the experimental 
results, which has been given earlier, it has become clear that pupils 
perform the pouring out skill almost perfectly, while their performance in 
volume measurement skill is not very good (obtained only 33% of the 
total marks). So, pupils' low achievement data obtained from the 
assessment method -p is perhaps the direct;, consequence, of .their poor 
performance in volume measurement, while the contribution of the 
pouring out skill on the final result can be considered as constant (all the 
pupils performed it very well). 

• ■ . > ■ 
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Table 2.1 

Achievement of pupils in carrying out 2 manipulative skills involved in 
experiment 2.1 



(The data-analyses were based on the fact that the 3 sets of pupils were assessed in 3 
sessions) 



Session 



Skill of measuring 
volume of liquid 



M 



M+P 



Skill of pouring out 
liquid 



P. 



M + P 



1+2 + 3 



X 


1.05 


1.30 


2.85 


2.80 


s.d. 


+0.86 


±0.90 


±0.36 


±0.40 


X 


1.00 


0.95 


3.00 


2.85 


s.d. 


+1.09 


±0.92 


±0.00 


±0.36 


X 


0.80 


0.90 


2.75 


2.75 


s.d. 


±0.93 


±0.94 } 


±0.43 


±0.43 


X 


0.95 


1.05 


2.87 


2.80 


s.d. 


+0.97 


+0.94 


+0.34 


+0.40 



x = mean value of achievement per pupil (maximum value is 3) 

s.d. = standard deviation in "x" 

M/P = only volume measurement/pouring out skill was assessed by a panel of 

judges in a session 
M + P/M + P_= Both the skills were assessed by a panel in a session 

(only the underlined skill assessment was used in calculation) 
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Table 2.2 

Achievement of pupils in carrying out 2 manipulative skills involved in 

experiment 2.1 

(The data analyses were based on the fact that 3 panels of judges carried out the 

assessment) 



Panel of judges 




Skill of 


measuring 


Skill of 


pouring out 






volume of 


liquid 


liquid 






M 




M + P 


P 


M + P 


A 


X 


1.05 




0.95 


2.75 


2.85 




s.d. 


±0.86 




±0.92 


±0.43 


±0.36 


B 


X 


1.00 




0.90 


2.85 


2.75 




s.d. 


. +1.09 




±0.94 -. 


±0.36 


±0.43 


C . 


X 


0.80 




1.30 


3.00 


2.80 




s.d. 


±0.93 




±0.90 


±0.0 


±0.40 


A + B + C 


X 


0.95 




1.05 


2.87 


2.80 




s.d. 


±0.97 




±0.94 


±0.34 


±0.40 



x = mean value of achievement per pupil (maximum value is 3) 

s.d. = standard deviation in "x" 

M/P = only volume measurement/pouring out skill was assessed by a panel of 

judges in a session 
M + P/M + P_= Both the skills were assessed by a panel of judges in a session 
(only the underlined skill assessment was used in calculation) 
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TABLE 2.3 

Amount of total agreement among the 3 judges in a panel in assessing the pupils! 

skill in experiment 2.1 

(The data analyses were based on the fact that the 3 sets of pupils were assessed in 3 

sessions) 



Session 




Skill of 


measuring 


Skill of 


pouring out 






volume of 


liquid 


liquid 






M 




M + p 


P 


M + P 


1 


X 


0.35 




0.30 


0.85 


0.85 




s.d. 


+0.48 




+0.46 


+0.36 


+0.40 


2 


X 


0.60 




0.45 


1.00 


0.85 




s.d. 


+0.49 




+0.50 


+0.00 


+0.36 


3 


X 


0.50 




0.50 


0.75 


0.75 




s.d. 


±0.50 




+0.50 


+0.43 


+0.43 


1+2 + 3 


X 


0.50 




> 
0.42 


0.87 


0.80 




s.d. 


+0.50 




+0.49 


+0.34 


+0.40 



x = mean value of agreement (maximum value is 1) 
s.d. = standard deviation in "x" 

M/P = only volume measurement/pouring out skill was assessed by a panel of 

i 
judges in a session. 

M + P/M + P = Both skills were assessed by a panel of judges in a session 

(only the underlined skill assessment was used in calculation) 
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Table 2.4 

Amount of total agreement among the 3 judges in a panel in assessing the pupils' 
skill in experiment 2.1 

'(The data analyses were" based on the fact that 3 panels of judges carried out the 
^assessment) 



panel of judges 




Skill of 


measuring 


Skill of 


pouring out 






volume of 


liquid 


liquid 






M 




NL+P 


P 


M + P 


A 


X 


0.35 




0.45 


0.75 


0.85 




s.d. 


+0.48 




+0.50 


+0.43 


+0.36 


B 


X 


0.60 




0.50 


0.85 


, 0.75 




s.d. 


+0.49 




+0.50 - 


. +0.36 


•••- ±0.43 


C 


X 


0.55 




0.30 


1.00 


0.85 




s.d. 


+0.50 




+0.46 


+0.36 


+0.40 


A + B + C 


X 


0.50 




> 
0.42 


. 0.87 


0.80 




s.d. 


+0.50 




+0.49 


+0.34 


+0.40 



: x = mean value of agreement (maximum value, is 1) 
rs.d. = standard deviation in "x" 
M/P = only volume measurement/pouring out skill was assessed by a panel of judges 

in a session 
M + P/M + P = Both skills were assessed by a panel of judges in a session, 
(only the underlined skill assessment was used in calculation) 
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Table 2.5 
sfcjjnary tables for several 2x3 ANOVA based on completely randomised groups for 
m&he<- reliability of assessment (agreement among the judges) of pupils' ability to 
f 'perform scientific manipulative skill in experiment 2 . 1 

/ d ) Measuring volume of liquid using measuring cylinder 



IB 



"Source of 
^variance 



SS df MS P F Significance of 

(obsv.) (crit) F (obsv.) 

atp=0.05. 



tlTl2 method gr.) 



0.209 
FES' (3 panels of judges) 0.517 

ILaX b ■ ■■■ ■",' °- 616 

||ror 28.450 

lotal 29.792 



1 


0.209 


0.837 


4.00 


n.s. (p { 0.05) 


2 


0.259 


1.040 


3.15 


n.s. (p <( 0.05) 


2 


0.308 


1.234 


.3.15 


n.s. (p ^ 0.05) 


114 


0.249 


.' - 


- : 




119 


_ 


_ 


_ 





"(b) Pouring liquid out of measuring cylinder 



|§ource of 
J§ariance 



SS 



df 



MS 



, F % / ?_,_, Significance of 
(obsv.) (crxt) , 

„ „- F (obsv.) 

at p=0.05 * ^^ av -' 



[TJ (2 method gr.) 0.133 

tM(3 panels of judges) 0.267 
ftXX B 0.467 

ffcor 15.800 



fi.~ 



F—Total 

mm 



16.667 



1 


0.133 


0.957 


4.00 


n.s. 


(p ^ G*'Q5) 


2 


0.134 


I 

0.960 


3.15 


n.s. 


(p ( 0.05) 


2 


0.234 


1.680 


3.15 


n.s. 


(p<0.05) 


114 


0.138 


- 


- 






119 


_ 


_ 


_ 







/continued 
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Table 2.5 (continued) 
if: Measuring volume of liquid using measuring cylinder 



W:<~ 

IIS 



Igource of 
^-Variance 



SS 



df ... 



MS 



' A (2 method gr. 

-.- -•■■ : " 

V B (3 sessions) 

^~&~ 
prior 

iTotal 



0.208 

1.067 

0.067 

28.450 

29.792 



1 

2 

2 

114 

119 



0.208 
0.533 
0.033 
0.249 



:|d[TPouring liquid out of measuring cylinder 



Source of 
■variance 



SS 



df 



MS 



A|( 2 method gr.] 
B 7(3 sessions) 

|x;b - 

.-Error 

Spa' 



0.133 

0.617 

0.117 

15.800 

16.667 



1 

2 

"2 

114 

119 



0.133 
0.308 
0.058 
0.138 



IBE 



fables 2.5 (a) - (d) : 



ft 



(obsv. 



(crit) 
at p=0.05 



Significance of 
F (obsv.) 



0.833 4.00 
2.138 3.15 
0.134 3.15 



n.s. (p /0.05) 
n.s. (p<^ 0.05) 
n.s. (p./ 0.05) 



F F 
(obsv.) (crit) 

at p=0.05 
> 



Significance of 
F (obsv.) 



0.957 4.00 
2.219 " 3.15 
0.421 3.15 



n.s. (p </ 0.05) 
n.s. (p <( 0.05) 
n.s. (p <( 0.05) 



S^tobsv.) 



(crit) 



n. 



denotes the value of F ratio obtained from the experimental 
results. 

denotes the critical value of F ratio at a given level of 
significance (i.e. p value) and at the appropriate df 
values (see reference 50 for the relevant statistical 
tables) . 

means "not significant". This refers to the fact that the 
value of observed F ratio is less than the critical value 
of F at p=0.05 and thus F (observed) is not significant at 
p /0.05 level. 



Table 2.6 
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Achievement of pupils (participating in Session 1) for carrying out 2 manipulative 
skills (measuring volume of liquid and pouring out liquid) involved in experiment 
2.1, assessed both by (1) Method -T (i.e. on the-spot observation of the performance of 
the techniques) and (2) Method -P (i.e. checking the reported results/final product of 
the experiment) " , " ' 

Achievement in assessment by Method -T Achievement 
in assessment by 



Method -P 



pupil No. Marks for measuring Marks for pouring liquid Marks for 

_ - vcLu-lTie, reported results 

M li+P Total marks P M+P Total marks 

(6max) (6max) (6max) 



■ 1 2 1 5 6 5 o , t. 

2 - - - - 

3 3 14 3 3 6 4 

4 2 2 2 3 5 4 

5 11 2 3 2 5 6 

6 2 3 5 3 3 6 5 

7 1 1 3 2 5 4 

8 10 1 3 3 6 1 
9 

10 1 1 2 3 3 6 3 

11 11 2 3 y 3 6 6 

12 • 1 1 2 2 4 5 

13 13 4 3 3 6 2 • 

14 1 2 3 3 3 6 4 

15 2 1 3 3 3 6 4 
16. 2 2 4 3 3 6 3 

17 0003 3 6 3 

18 03 3 2 2 4 5 . 

19 1 1 2 3 ' 3 6 4 

20 1 2 3 3 3 6 4 

21 1 1 3 3 6 3 
22 

23 1 1 3 3 6 1 

M/Pj Only volume measurement/pouring out skill was assessed (max. marks for each 
skill = 3) 

M + P/M + P: Both the skills were assessed, but only the underlined skill's mark is 
involved in the calculation. 



Table 2.7 
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Achievement of pupils (participating in Session 2 ) for carrying out 2 manipulative 
skills (measuring volume of liquid and pouring out liquid) involved in experiment 
2.1, assessed both by (1) Method -T (i.e. on-the-spot observation of the performance of 
the techniques)- and (2) Method -P (i,e, checking the reported results /final product of 
the experiment). 



pupil No. 



Achievement in assessment by Method -T 



Marks for measuring 
volume 



Marks for pouring liquid 



M 



M+P 



Total marks 
(6 max) 



M+P 



Total marks 
(6 max) 



Achievement 

in assessment 

by Method -P 

Marks for 

reported 

results 

(6 max) 



> ■ 1 


2 


2 


1 


"3 





4 





5 





6 





7 


1 


8 


1 


9 





10 





11 


1 


12 


2 


-;:■ 13 


1 


: ■ 14 


3 


15 


2 


16 





17 


3 


18 


3 


19 





20 






1 




1 



1 
1 
1 





1 
1 

2 
2 
1 
3 
3 
1 




3 
1 

1 

1 
2 
2 


1 
3 
2 
5 
4 
1 
6 
6 
1 




3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 



3 
3 
3 
3 
2 
3 
3 
3 
3 
3 
3 
3 
3 
2 
3 
3 
3 
3 
3 
2 



6 
6 
6 
6 
5 
6 
6 
6 
6 
6 
6 
.6 
6 
5 
6 
6 
6 
6 
6 
5 



3 
6 

4 
4 
5 
4 
1 
3 
6 
3 
4 
5 
3 
5 
5 
4 
2 
4 
3 



M/P: Only volume measurement/ pouring out skill was assessed (max. marks for 
each skill = 3) 

M.+ P/M + P: Both the skills were assessed, but only the underlined skill's mark is 
involved in the calculation. 
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Achievement of pupils (participating in Session 3 ) for carrying out 2 manipulative 
skills (measuring volume of liquid and pouring out liquid) involved in experiment 
2.1/ assessed both by (1) Method -T (i.e. on-the-spot observation of the performance of 
the techniques) and (2) Method -P (i.e. checking the reported results/final product of 
the experiment). 



[:"•- .:.-■- 




Achievement in Assessment by Method - 


T 




Achievement in 


(■•• "" 
















assesmenthy 


f : '~ ■■■■■ 

LV- '• ■ 

L-... " 

L'" ■' "•• 
















Method -P 


^ Pupil No. 


Marks for measuring 






Marks for pouring 




Marks for 


1. 
[. - ■ ■ . 




volume 






liquid 




reported results 


f. . 


M 


M+P 


Total marks 


p 


M+P 


Total marks 




I' 






(6 max) 








(6 max) 


(6 max) 


!■ 1 





1 


1 


3 




3 


6 


4 


! " ' 2 





1. 


1 


3 




3 


6 


4 


r 3 '. 











3 




2 


5 


1 


'■' 4 


1 


3 


4 


3 




3 


6 


5 


5 











2 




2 


4 


5 


6 


1 





1 


3 




3 


6 


6 


7 











2 


> 


2 


4 


4 


8 


2 


2 


4 


2 




2 


4 


2 


9 


2 


2 


4 


2 




3 


5 


4 


10. 


2 


2 


4 


3 




3 


6 


5 


11 


3 


2 


5 


3 




3 


6 


5 


■::'u 12 





1 


1 


3 




3 


6 


6 


:V..::""i' : i3 


1 





1 


3 




3 


6 


2 


14 


1 


1 


2 


3 


l 


3 


6 


4 


15 


2 


2 


4 


3 




3 


6 


3 


:■■ 16 











3 




3 


6 





: 17 


1 





1 


3 




2 


5 


5 


18 





1 


1 


2 




3 


5 


3 


19 











3 




3 


6 


4 


20 











3 




3 


6 






M/P: Only volume measurement/pouring out skill was assessed (max. marks for 
each skill = 3) 

M_+ P/M + P: Both the skills were assessed, but only the underlined skill's mark is 
involved in the calculation. 



54 



Table 2.9 



EL 

m 

IL 



^fituation of total agreement ("1") /disagreement ("0") among -3 judges 
M, a panel during the assessment in session 1 . 

fltipil No. ■ M by panel A P by .panel B M + P_ by panel C 



m 



i 

2 
3 

6 

#T7 



10 

Pi 2' 

"13 

ftl4 
.-15 

Si" 17. 

^18' 

% 19 
20 

■ - 21 



22 

pJL23 
IS 



1 




1 




1 





1 
1 




1 



1 



1 
1 
1 
1 

1 
1 



1 
1 
1 
1 
1 



1 
1 
1 






1 


1 


1 








1 


1 








1 


1 





1 





1 





-0 


1 


1 





1 





1 





1 


1 


1 


1 








1 





1 





1 



*M : only "Measurement of volume" skill assessed 



r£. : only "Pouring out liquid" skill assessed 



,.H + P : Both the.- skills assessed 
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Table 2.10 



Situation of total agreement (" 1 ") /disagreement ("0") among 3 judges 
in a panel during the assessment in session 2 . 

pupil No. . .. M by panel B P by panel C M + . P by panel A 



1 i 

2 i 

3 1 l 

4 1 l 
5-1- 1 

6 1 l 

7 l 

8 l 

9 1 1 
10 . .1 i 

11 . ; ■ 1 

12 l 

13 l 

14 1 i 

15 1 

16 1 i 

17 1 1 

18 1 i 

19 1 i 

20 l ! 



M : only "Measurement of volume" skill assessed 



£ : only "Pouring out liquid" skill assessed 
M + P : Both the skills assessed 






1 


1 


1 


1 


1 





1 


1 








1 





1 





1 


1 


1 


1 


1 


1 


1 





1 





1 











1 





1 


i 


1 


l 


1 





1 


1 


1 



h: 
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Table 2.11 

•tuation of total agreement ("1") /disagreement ("0") among 3 judges 
. a panel during the assessment in session 3. 



pupil No'. M by panel C ■■■-■• j? by panel A M + ' P_ by panel B' 



1 

W ■ '; 2 

.3 



- 



- .4 , ° 

"5 - 1 

. :'6 

7 l' 

8. 

9 



.10 ■ 

11 1 

12 1 

13 

14 

15 

16 1 

17 

18 1 

19 1 

20 1 



1 





1 


1 





1 


1 


1 





1 


1 


1 





1 





1 


1 


1 





1 




















1 


1 





1 


1 





1 


1 





1 


1 


1 . 


1 


1 





1 


1 > 





1 


1 


1 


1 


1 


1 











1 


1 


1 


1 


1 


1 


1 



r. " 



Li M : only "Measurement of volume" skill assessed 
;"-:£, : only "Pouring out liquid" skill assessed 



' ' ' . 






;•;: H + P. : Both the skills assessed 
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Table 2.1.2 

pearson-r (Pearson product moment correlation coefficient) between the 
achievements of pupils after being assessed by observation of their performance 
(Method -T) and that after being assessed by checking the reported results of the 
experiment (Method -P) for two practical skills (a) measurement of volume of liquid 
and (b) pouring out liquid, in the experiment 2.1 

Pouring out 
liquid 



0.078 
-0.011 
0.111 
-0.044 
-0.183 
0.021 
-0.104 



No. 


of run of 




Measurement of volume 


the 


experiment 

1st 

2nd 

3rd 

4th 

5th 

6th 






of liquid 

0.159 
-0.160 
0.223 
-0.023 
0.100 
0.306 


Combined runs 


1-6, 




0.142 


i.e. i 


or the whole e 


xperimental work 





Results of all the samples, i.e. pupils participating in 3 sessions, were used in the 
; . 7 Pearson-r value calculation for each run of the experiment) 






£■■■■ , 



. 
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Fig. 2.1 Plots of achievement in assessment (on-the-spot 

observation) by Method -T for measuring volume by 
the pupils participating in 3 sessions against their 
achievement in assessment by Method -P (checking the 
reported results of the experiment) for carrying out 
the whole experiment 2.1. 



(The data are in Tables 2,6, 2.7 and 2, 
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MPS 
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o 

+J 
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,0 
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Fig 2.2 Plots of achievement in assessment (on-the-spot 

observation) by Method -T for pouring out liquid by 
the pupils participating in .3 sessions against their 
achievement in assessment by Method -P (checking the 
--reported results of the experiment) for carrying out 
the whole experiment 2.1. 



(The data are in Tables 2.6, 2.7 and 2.8) 



